A Step toward Robot Teleoperation with Eyes and Hand

Despite the continuous improvement of master interfaces, distant robot teleoperation remains a challenging task. In many applications (e.g. spaceships, underwater or flying drones, robotic arms that operate in hazardous conditions in factories), the operator has only an indirect vision of the remote environment, provided by a video camera usually mounted on the robot end-effector itself, and displayed on a 2D monitor. Whereas any controller is capable of planning a path to follow a prescribed 3D trajectory, driving such an eye-in-hand robot in real time is known to be difficult. Even a skilled user will not make the most of a 6-axis master interfaces that could normally induce a smooth and continuous motion. Instead, she/he will likely generate a succession of independent translations and rotations punctuated by frequent stops. The main reason is that the target (e.g. object to grasp) must be kept inside the camera field of view during robot motion. It requires to combine translation... Document type : Communication à un colloque (Conference Paper) Référence bibliographique Despinoy, Fabien ; Vitrani, Marie-Aude ; Herman, Benoît. A step toward robot teleoperation with eyes and hand.5th International Workshop on Human-Friendly Robotics (HFR2012) (Brussels, Belgium, du 18/10/2012 au 19/10/2012). A Step toward Robot Teleoperation with Eyes and Hand Fabien Despinoy, Marie-Aude Vitrani, and Benoı̂t Herman Despite the continuous improvement of master interfaces, distant robot teleoperation remains a challenging task. In many applications (e.g. spaceships, underwater or flying drones, robotic arms that operate in hazardous conditions in factories), the operator has only an indirect vision of the remote environment, provided by a video camera usually mounted on the robot end-effector itself, and displayed on a 2D monitor. Whereas any controller is capable of planning a path to follow a prescribed 3D trajectory, driving such an eye-in-hand robot in real time is known to be difficult. Even a skilled user will not make the most of a 6-axis master interfaces that could normally induce a smooth and continuous motion. Instead, she/he will likely generate a succession of independent translations and rotations punctuated by frequent stops. The main reason is that the target (e.g. object to grasp) must be kept inside the camera field of view during robot motion. It requires to combine translations and rotations of the end-effector, with a ratio between linear and angular velocities that depends on the distance to the target—the latter being usually unknown. On the other hand, one can assume that the user will look most of the time at the target on the video monitor. Our idea is to gather additional information on the remote target with an eye-tracking device, and to use it to facilitate teleoperation. A first possibility is to estimate the distance between endeffector and target, so as to make the robot rotate around the target instead of around its end-effector. This can be achieved iteratively by measuring the user’s eye motion induced by a robot displacement. A second possibility is to maintain the target at the center of the screen, either by correcting automatically the end-effector orientation during a manual translation, or conversely, by shifting the robot when the user rotates it. In these two similar scenarios, the user has to control only 4 degrees of freedom manually, the remaining two being handled by the controller thanks to the eye-tracker 2D signal. A proof-of-concept prototype was built around a 6-axis Adept Viper s650 robot (see Fig. 1). The teleoperation system comprises a webcam mounted on the robot end-effector, a head-mounted eye-tracking camera (EYE-TRAC H6 from Applied Science Laboratories, Bedford, MA), a stand to prevent any head motion with respect to the monitor where the webcam images are displayed, and a 6-axis joystick (SpaceNavigator from 3Dconnexion, Waltham, MA). This work was supported by Fondation Belge Contre le Cancer (Grant No. SCIE 2006/20), and by own funds from UCL, ISIR, and F.R.S.-FNRS. F. Despinoy and B. Herman are with the Center for Reserach in Mechatronics, Université catholique de Louvain, Louvain-la-Neuve, Belgium. F. Despinoy is also with Polytech Paris–UPMC, Université Pierre et Marie Curie, Paris, France. B . Herman is also with Fonds de la Recherche Scientifique – FNRS. M.-A. Vitrani is with Institut des Systèmes Intelligents et de Robotique, Université Pierre et Marie Curie – CNRS (UMR7222), Paris, France. Corresponding author: [email protected] Viper robot USB webcam